# Lightweight LLaMA

A fine-tuned language model based on the 4-bit quantized version of TinyLLaMA, efficiently trained using Unsloth and TRL libraries

Sheared-LLaMA-1.3B-Pruned is a 1.3B parameter model pruned from Llama-2-7b, without continued pretraining, primarily used for studying pruning techniques and their impacts.